Customizable model for throttling and prioritizing orders in a cloud environment

ABSTRACT

A method for controlling the provisioning of a service associated with a customer order is disclosed. The method may comprise receiving, by a cloud infrastructure system, an order for a service from a plurality of service provided by the cloud infrastructure system, the cloud infrastructure system comprising one or more computing devices. Additionally, the method may comprise determining, by a computing device from the one or more computing devices, based upon a set of rules and system-related information, whether the order is to be forwarded for provisioning one or more resources for enabling the order. Then, based upon the determining, the method may forward the order for provisioning of the one or more resources or not forwarding the order for provisioning.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a non-provisional of and claims the benefitand priority under 35 U.S.C. 119(e) of the following applications, theentire contents of which are incorporated herein by reference for allpurposes:

-   (1) U.S. Provisional Application No. 61/698,413, filed Sep. 7, 2012,    entitled TENANT AUTOMATION SYSTEM;-   (2) U.S. Provisional Application No. 61/698,459, filed Sep. 7, 2012,    entitled SERVICE DEVELOPMENT INFRASTRUCTURE;-   (3) U.S. Provisional Application No. 61/785,299, filed Mar. 14,    2013, entitled CLOUD INFRASTRUCTURE;-   (4) U.S. Provisional Application No. 61/798,714, filed Mar. 15,    2013, entitled CUSTOMIZABLE MODEL FOR THROTTLING AND PRIORITIZING    ORDERS IN A CLOUD ENVIRONMENT; and-   (5) U.S. Provisional Application No. 61/794,427, filed Mar. 15,    2013, entitled CLOUD INFRASTRUCTURE.

BACKGROUND

The present disclosure relates to computer systems and software, andmore particularly to techniques for facilitating and automating theprovision of services in a cloud environment.

Cloud computing is a model for enabling convenient, on-demand networkaccess to a shared pool of configurable computing resources (e.g.,networks, servers, storage, applications, and services). The servicesprovided or accessed through the cloud (or network) are referred to ascloud services. There is a lot of processing that needs to be performedby a cloud service provider to make cloud services available to asubscribing customer. Due to its complexity, much of this processing isstill done manually. For example, provisioning resources for providingsuch cloud services can be a very labor intensive process.

SUMMARY

This summary is not intended to identify key or essential features ofthe claimed subject matter, nor is it intended to be used in isolationto determine the scope of the claimed subject matter. The subject mattershould be understood by reference to appropriate portions of the entirespecification of this patent, any or all drawings and each claim.

According to some embodiments, a method for controlling the provisioningof a service associated with a customer order is disclosed. The methodmay comprise receiving, by a cloud infrastructure system, an order for aservice from a plurality of service provided by the cloud infrastructuresystem, the cloud infrastructure system comprising one or more computingdevices. Additionally, the method may comprise determining, by acomputing device from the one or more computing devices, based upon aset of rules and system-related information, whether the order is to beforwarded for provisioning one or more resources for enabling the order.Then, based upon the determining, the method may forward the order forprovisioning of the one or more resources or not forwarding the orderfor provisioning.

According to some embodiments, a system comprising: one or morecomputing devices configurable to offer a set of cloud services; memoryconfigurable to store an order for a service from the set of cloudservice provided by the cloud infrastructure system; and wherein acomputing device from the one or more computing devices configurable to:determine based upon a set of rules and system-related information,whether the order is to be forwarded for provisioning one or moreresources for enabling the order; and based upon the determination,forward the order for provisioning of the one or more resources or notforwarding the order for provisioning.

According to some embodiments, one or more computer-readable mediastoring computer executable instructions for a cloud infrastructuresystem configured to offer a set of cloud services that, when executed,cause one or more computing devices in the cloud infrastructure to:store an order for a service from the set of cloud service provided bythe cloud infrastructure system; determine based upon a set of rules andsystem-related information, whether the order is to be forwarded forprovisioning one or more resources for enabling the order; and basedupon the determination, forward the order for provisioning of the one ormore resources or not forwarding the order for provisioning.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present invention are described indetail below with reference to the following drawing figures:

FIG. 1A is a logical view of a cloud infrastructure system according toone embodiment of the present invention.

FIG. 1B is a simplified block diagram of a hardware/software stack thatmay be used to implement a cloud infrastructure system according to anembodiment of the present invention.

FIG. 2 is a simplified block diagram of a system environment forimplementing the cloud infrastructure system shown in FIG. 1A.

FIG. 3A depicts a simplified flowchart 300 depicting processing that maybe performed by the TAS module in the cloud infrastructure system, inaccordance with an embodiment of the present invention.

FIG. 3B depicts a simplified high level diagram of one or moresub-modules in the TAS module in the cloud infrastructure system, inaccordance with an embodiment of the present invention.

FIG. 4 depicts an exemplary distributed deployment of the TAS component,according to an embodiment of the present invention.

FIG. 5 is a simplified block diagram illustrating the interactions ofthe SDI module with one or more modules in the cloud infrastructuresystem, in accordance with an embodiment of the present invention.

FIG. 6 depicts a simplified high level diagram of sub-modules of the SDImodule according to an embodiment of the present invention.

FIG. 7A depicts a simplified flowchart depicting processing that may beperformed by the SDI component in the cloud infrastructure system, inaccordance with an embodiment of the present invention.

FIG. 7B depicts a simplified block diagram showing the high-levelarchitecture of a Nuviaq system 710 and its relationships with othercloud infrastructure components according to an embodiment of thepresent invention.

FIG. 7C depicts an example sequence diagram illustrating steps of aprovisioning process using a Nuviaq system according to an embodiment ofthe present invention.

FIG. 7D depicts an example sequence diagram illustrating steps of adeployment process using a Nuviaq system according to an embodiment ofthe present invention.

FIG. 7E depicts an example of database instances provisioned for adatabase service according to an embodiment of the present invention.

FIG. 8 depicts a simplified high level diagram of the components in theTAS module that may incorporate aTAS order flow control module,according to some embodiments.

FIG. 9 depicts a simplified flowchart depicting processing that may beperformed by the TAS flow control module 805 in the cloud infrastructuresystem, in accordance with an embodiment of the present invention.

FIG. 10 depicts a simplified high level diagram of the components in TASorder flow control module, according to some embodiments.

FIG. 11 depicts a simplified flowchart depicting processing that may beperformed by the TAS flow control module 805 in the cloud infrastructuresystem, in accordance with another embodiment of the present invention.

FIG. 12 is a simplified block diagram of a computing system that may beused in accordance with embodiments of the present invention.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofembodiments of the invention. However, it will be apparent that variousembodiments may be practiced without these specific details. The figuresand description are not intended to be restrictive.

Certain embodiments of the present invention provide techniques forautomating the provisioning, managing and tracking of services providedby a cloud infrastructure system.

In certain embodiments, a cloud infrastructure system may include asuite of applications, middleware and database service offerings thatare delivered to a customer in a self-service, subscription-based,elastically scalable, reliable, highly available, and secure manner. Anexample of such a cloud infrastructure system is the Oracle Public Cloudprovided by the present assignee.

A cloud infrastructure system may provide many capabilities including,but not limited to, provisioning, managing and tracking a customer'ssubscription for services and resources in the cloud infrastructuresystem, providing predictable operating expenses to customers utilizingthe services in the cloud infrastructure system, providing robustidentity domain separation and protection of a customer's data in thecloud infrastructure system, providing customers with a transparentarchitecture and control of the design of the cloud infrastructuresystem, providing customers assured data protection and compliance withdata privacy standards and regulations, providing customers with anintegrated development experience for building and deploying services inthe cloud infrastructure system and providing customers with a seamlessintegration between business software, middleware, database andinfrastructure services in the cloud infrastructure system.

In certain embodiments, services provided by the cloud infrastructuresystem may include a host of services that are made available to usersof the cloud infrastructure system on demand such as online data storageand backup solutions, Web-based e-mail services, hosted office suitesand document collaboration services, database processing, managedtechnical support services and the like. Services provided by the cloudinfrastructure system can dynamically scale to meet the needs of itsusers. A specific instantiation of a service provided by cloudinfrastructure system is referred to herein as a service instance. Ingeneral, any service made available to a user via a communicationnetwork such as the Internet from a cloud service provider's system isreferred to as a cloud service. Typically, in a public cloudenvironment, servers and systems that make up the cloud serviceprovider's system are different from the customer's own on-premisesservers and systems. For example, a cloud service provider's system mayhost an application and a user may, via a communication network such asthe Internet, on demand, order and use the application.

A service in a computer network cloud infrastructure includes protectedcomputer network access to storage, a hosted database, a hosted webserver, a software application, or other service provided by a cloudvendor to a user, or as otherwise known in the art. For example, aservice can include password-protected access to remote storage on thecloud through the Internet. As another example, a service can include aweb service-based hosted relational database and script-languagemiddleware engine for private use by a networked developer. As anotherexample, a service can include access to an email software applicationhosted on a cloud vendor's web site.

FIG. 1A is a logical view of a cloud infrastructure system according toone embodiment of the present invention. Cloud infrastructure system 100may provide a variety of services via a cloud or networked environment.These services may include one or more services provided under Softwareas a Service (SaaS) category, Platform as a Service (PaaS) category,Infrastructure as a Service (IaaS) category, or other categories ofservices including hybrid services. A customer, via a subscriptionorder, may order one or more services provided by cloud infrastructuresystem 100. Cloud infrastructure system 100 then performs processing toprovide the services in the customer's subscription order.

Cloud infrastructure system 100 may provide the cloud services viadifferent deployment models. For example, services may be provided undera public cloud model where cloud infrastructure system 100 is owned byan organization selling cloud services (e.g., owned by Oracle) and theservices are made available to the general public or different industryenterprises. As another example, services may be provided under aprivate cloud model where cloud infrastructure system 100 is operatedsolely for a single organization and may provide services for one ormore entities within the organization. The cloud services may also beprovided under a community cloud model where cloud infrastructure system100 and the services provided by system 100 are shared by severalorganizations in a related community. The cloud services may also beprovided under a hybrid cloud model, which is a combination of two ormore different models.

As shown in FIG. 1A, cloud infrastructure system 100 may comprisemultiple components, which working in conjunction, enable provision ofservices provided by cloud infrastructure system 100. In the embodimentillustrated in FIG. 1A, cloud infrastructure system 100 includes a SaaSplatform 102, a PaaS platform 104, an IaaS platform 110, infrastructureresources 106, and cloud management functionality 108. These componentsmay be implemented in hardware, or software, or combinations thereof.

SaaS platform 102 is configured to provide cloud services that fallunder the SaaS category. For example, SaaS platform 102 may providecapabilities to build and deliver a suite of on-demand applications onan integrated development and deployment platform. SaaS platform 102 maymanage and control the underlying software and infrastructure forproviding the SaaS services. By utilizing the services provided by SaaSplatform 102, customers can utilize applications executing on cloudinfrastructure system 100. Customers can acquire the applicationservices without the need for customers to purchase separate licensesand support.

Various different SaaS services may be provided. Examples includewithout limitation services that provide solutions for sales performancemanagement, enterprise integration and business flexibility for largeorganizations, and the like. In one embodiment, the SaaS services mayinclude Customer Relationship Management (CRM) services 110 (e.g.,Fusion CRM services provided by the Oracle cloud), Human CapitalManagement (HCM)/Talent Management services 112, and the like. CRMservices 110 may include services directed to reporting and managementof a sales activity cycle to a customer, and others. HCM/Talent services112 may include services directed to providing global workforcelifecycle management and talent management services to a customer.

Various different PaaS services may be provided by PaaS platform 104 ina standardized, shared and elastically scalable application developmentand deployment platform. Examples of PaaS services may include withoutlimitation services that enable organizations (such as Oracle) toconsolidate existing applications on a shared, common architecture, aswell as the ability to build new applications that leverage the sharedservices provided by the platform. PaaS platform 104 may manage andcontrol the underlying software and infrastructure for providing thePaaS services. Customers can acquire the PaaS services provided by cloudinfrastructure system 100 without the need for customers to purchaseseparate licenses and support. Examples of PaaS services include withoutlimitation Oracle Java Cloud Service (JCS), Oracle Database CloudService (DBCS), and others.

By utilizing the services provided by PaaS platform 104, customers canutilize programming languages and tools supported by cloudinfrastructure system 100 and also control the deployed services. Insome embodiments, PaaS services provided by the cloud infrastructuresystem 100 may include database cloud services 114, middleware cloudservices (e.g., Oracle Fusion Middleware services) 116 and Java cloudservices 117. In one embodiment, database cloud services 114 may supportshared service deployment models that enable organizations to pooldatabase resources and offer customers a database-as-a-service in theform of a database cloud, middleware cloud services 116 provides aplatform for customers to develop and deploy various businessapplications and Java cloud services 117 provides a platform forcustomers to deploy Java applications, in the cloud infrastructuresystem 100. The components in SaaS platform 102 and PaaS platform 104illustrated in FIG. 1A are meant for illustrative purposes only and arenot intended to limit the scope of embodiments of the present invention.In alternate embodiments, SaaS platform 102 and PaaS platform 104 mayinclude additional components for providing additional services to thecustomers of cloud infrastructure system 100.

Various different IaaS services may be provided by IaaS platform 110.The IaaS services facilitate the management and control of theunderlying computing resources such as storage, networks, and otherfundamental computing resources for customers utilizing servicesprovided by the SaaS platform and the PaaS platform.

In certain embodiments, cloud infrastructure system 100 includesinfrastructure resources 106 for providing the resources used to providevarious services to customers of the cloud infrastructure system 100. Inone embodiment, infrastructure resources 106 includes pre-integrated andoptimized combinations of hardware such as servers, storage andnetworking resources to execute the services provided by the PaaSplatform and the SaaS platform.

In certain embodiments, cloud management functionality 108 providescomprehensive management of cloud services (e.g., SaaS, PaaS, IaaSservices) in the cloud infrastructure system 100. In one embodiment,cloud management functionality 108 includes capabilities forprovisioning, managing and tracking a customer's subscription receivedby the cloud infrastructure system 100, and the like.

FIG. 1B is a simplified block diagram of a hardware/software stack thatmay be used to implement cloud infrastructure system 100 according to anembodiment of the present invention. It should be appreciated thatimplementation depicted in FIG. 1B may have other components than thosedepicted in FIG. 1B. Further, the embodiment shown in FIG. 1B is onlyone example of a cloud infrastructure system that may incorporate anembodiment of the invention. In some other embodiments, cloudinfrastructure system 100 may have more or fewer components than shownin FIG. 1B, may combine two or more components, or may have a differentconfiguration or arrangement of components. In certain embodiments, thehardware and software components are stacked so as to provide verticalintegration that provides optimal performance.

Various types of users may interact with cloud infrastructure system100. These users may include, for example, end users 150 that caninteract with cloud infrastructure system 100 using various clientdevices such as desktops, mobile devices, tablets, and the like. Theusers may also include developers/programmers 152 who may interact withcloud infrastructure system 100 using command line interfaces (CLIs),application programming interfaces (APIs), through various integrateddevelopment environments (IDEs), and via other applications. User mayalso include operations personnel 154. These may include personnel ofthe cloud service provider or personnel of other users.

Application services layer 156 identifies various cloud services thatmay be offered by cloud infrastructure system 100. These services may bemapped to or associated with respective software components 160 (e.g.,Oracle WebLogic server for providing Java services, oracle database forproviding database services, and the like) via a service integration andlinkages layer 158.

In certain embodiments, a number of internal services 162 may beprovided that are shared by different components or modules of cloudinfrastructure system 100 and by the services provided by cloudinfrastructure system 100. These internal shared services may include,without limitation, a security and identity service, an integrationservice, an enterprise repository service, an enterprise managerservice, a virus scanning and white list service, a high availability,backup and recovery service, service for enabling cloud support in IDEs,an email service, a notification service, a file transfer service, andthe like.

Runtime infrastructure layer 164 represents the hardware layer on whichthe various other layers and components are built. In certainembodiments, runtime infrastructure layer 164 may comprise one Oracle'sExadata machines for providing storage, processing, and networkingresources. An Exadata machine may be composed of various databaseservers, storage Servers, networking resources, and other components forhosting cloud-services related software layers. In certain embodiments,the Exadata machines may be designed to work with Oracle Exalogic, whichis an engineered system providing an assemblage of storage, compute,network, and software resources. The combination of Exadata and Exalogicprovides a complete hardware and software engineered solution thatdelivers high-performance, highly available, scalable, secure, and amanaged platform for providing cloud services.

FIG. 2 is a simplified block diagram of a system environment forimplementing the cloud infrastructure system shown in FIG. 1A accordingto an embodiment of the present invention. In the illustratedembodiment, system environment 230 includes one or more client computingdevices 224, 226 and 228 that may be used by users to interact withcloud infrastructure system 100. A client device may be configured tooperate a client application such as a web browser, a proprietary clientapplication (e.g., Oracle Forms), or some other application, which maybe used by a user of the client device to interact with cloudinfrastructure system 100 to utilize services provided by cloudinfrastructure system 100.

It should be appreciated that cloud infrastructure system 100 depictedin FIG. 2 may have other components than those depicted in FIG. 2.Further, the embodiment shown in FIG. 2 is only one example of a cloudinfrastructure system that may incorporate an embodiment of theinvention. In some other embodiments, cloud infrastructure system 100may have more or fewer components than shown in FIG. 2, may combine twoor more components, or may have a different configuration or arrangementof components.

Client computing devices 224, 226 and 228 may be general purposepersonal computers (including, by way of example, personal computersand/or laptop computers running various versions of Microsoft Windowsand/or Apple Macintosh operating systems), cell phones or PDAs (runningsoftware such as Microsoft Windows Mobile and being Internet, e-mail,SMS, Blackberry, or other communication protocol enabled), workstationcomputers running any of a variety of commercially-available UNIX orUNIX-like operating systems (including without limitation the variety ofGNU/Linux operating systems), or any other computing device. Forexample, client computing devices 224, 226 and 228 may be any otherelectronic device, such as a thin-client computer, Internet-enabledgaming system, and/or personal messaging device, capable ofcommunicating over a network (e.g., network 232 described below).Although exemplary system environment 230 is shown with three clientcomputing devices, any number of client computing devices may besupported. Other devices such as devices with sensors, etc. may interactwith cloud infrastructure system 100.

A network 232 may facilitate communications and exchange of data betweenclients 224, 226 and 228 and cloud infrastructure system 100. Network232 may be any type of network familiar to those skilled in the art thatcan support data communications using any of a variety ofcommercially-available protocols, including without limitation TCP/IP,SNA, IPX, AppleTalk, and the like. Merely by way of example, network 232can be a local area network (LAN) such as an Ethernet network, aToken-Ring network and/or the like, a wide-area network, a virtualnetwork, including without limitation a virtual private network (VPN),the Internet, an intranet, an extranet, a public switched telephonenetwork (PSTN), an infra-red network, a wireless network (e.g., anetwork operating under any of the IEEE 802.1x suite of protocols, theBluetooth protocol known in the art, and/or any other wirelessprotocol), and/or any combination of these and/or other networks.

Cloud infrastructure system 100 may comprise one or more computersand/or servers which may be general purpose computers, specializedserver computers (including, by way of example, PC servers, UNIXservers, mid-range servers, mainframe computers, rack-mounted servers,etc.), server farms, server clusters, or any other appropriatearrangement and/or combination. The computing devices that make up cloudinfrastructure system 100 may run any of operating systems or a varietyof additional server applications and/or mid-tier applications,including HTTP servers, FTP servers, CGI servers, Java servers, databaseservers, and the like. Exemplary database servers include withoutlimitation those commercially available from Oracle, Microsoft, Sybase,IBM and the like.

In various embodiments, cloud infrastructure system 100 may be adaptedto automatically provision, manage and track a customer's subscriptionto services offered by cloud infrastructure system 100. In oneembodiment, as depicted in FIG. 2, the components in cloudinfrastructure system 100 include an Identity Management (IDM) module200, a services module 202, a Tenant Automation System (TAS) module 204,a Service Deployment Infrastructure (SDI) module 206, an EnterpriseManager (EM) module 208, one or more front-end web interfaces such as astore user interface (UI) 210, a cloud user interface (UI) 212, and asupport user interface (UI) 216, an order management module 214, salespersonnel 218, operator personnel 220 and an order database 224. Thesemodules may include or be provided using one or more computers and/orservers which may be general purpose computers, specialized servercomputers, server farms, server clusters, or any other appropriatearrangement and/or combination. In one embodiment, one or more of thesemodules can be provided by cloud management functionality 108 or IaaSplatform 110 in cloud infrastructure system 100. The various modules ofthe cloud infrastructure system 100 depicted in FIG. 2 are meant forillustrative purposes only and are not intended to limit the scope ofembodiments of the present invention. Alternative embodiments mayinclude more or fewer modules than those shown in FIG. 2.

In an exemplary operation, at (1) a customer using a client device suchas client device 224 or 226 may interact with cloud infrastructuresystem 100 by browsing the various services provided by cloudinfrastructure system 100 and placing an order for a subscription forone or more services offered by cloud infrastructure system 100. Incertain embodiments, the customer may access store UI 210 or cloud UI212 and place a subscription order via these user interfaces.

The order information received by cloud infrastructure system 100 inresponse to the customer placing an order may include informationidentifying the customer and one or more services offered by the cloudinfrastructure system 100 that the customer intends to subscribe to. Asingle order may include orders for multiple services. For instance, acustomer may login to cloud UI 212 and request a subscription for a CRMservice and a Java cloud service in the same order.

Additionally, the order may also include one or more service levels forthe ordered services. As used herein, and as will be discussed ingreater detail below, a service level for a service determines theamount of resources to be allocated for providing the requested servicein the context of the subscription, such as the amount of storage,amount of computing resources, data transfer facilities, and the like.For example, a basic service level may provide a minimum level ofstorage, data transmission, or number of users, and higher servicelevels may include additional resources.

In addition, in some instances, the order information received by cloudinfrastructure system 100 may include information indicative of acustomer level, and the time period during which the service is desired.The customer level specifies the priority of the customer making thesubscription request. In one example, the priority may be determinedbased on the quality of service that the cloud infrastructure system 100guarantees or promises the customer as specified by a Service LevelAgreement (SLA) agreed to between the customer and the provider of thecloud services. In one example, the different customer levels include abasic level, a silver level and a gold level. The time period for aservice may specify the start date and time for the service and the timeperiod for which the service is desired (e.g., a service end date andtime may be specified).

In one embodiment, a customer may request a new subscription via storeUI 210 or request for a trial subscription via cloud UI 212. In certainembodiments, store UI 210 may represent the service provider's eCommercestore front (e.g., www.oracle.com/store for Oracle Cloud services).Cloud UI 212 may represent a business interface for the serviceprovider. Consumer can explore available services and sign up forinterested services through cloud UI 212. Cloud UI 212 captures userinput necessary for ordering trial subscriptions provided by cloudinfrastructure system 100. Cloud UI 212 may also be used to view accountfeatures and configure the runtime environment located within cloudinfrastructure system 100. In addition to placing an order for a newsubscription, store UI 210 may also enable the customer to perform othersubscription-related tasks such as changing the service level of asubscription, extending the term of the subscription, increasing theservice level of a subscription, terminating an existing subscription,and the like.

After an order has been placed per (1), at (2), the order informationthat is received via either store UI 210 or cloud UI 212 is stored inorder database 224, which can be one of several databases operated bycloud infrastructure system 100 and utilized in conjunction with othersystem elements. While order database 224 is shown logically as a singledatabase in FIG. 2, in actual implementation, this may comprise one ormore databases.

At (3), the order is forwarded to order management module 214. Ordermanagement module 214 is configured to perform billing and accountingfunctions related to the order such as verifying the order and uponverification, booking the order. In certain embodiments, ordermanagement module 214 may include a contract management module and aninstall base module. The contract management module may store contractinformation associated with the customer's subscription order such asthe customer's service level agreement (SLA) with cloud infrastructuresystem 100. The install base module may include detailed descriptions ofthe services in the customer's subscription order. In addition to orderinformation, the install base module may track installation detailsrelated to the services, product status and support service historyrelated to the services. As a customer orders new services or upgradesexisting ones, the install base module may automatically add new orderinformation.

At (4), information regarding the order is communicated to TAS module204. In one embodiment, TAS module 204 utilizes the order information toorchestrate the provisioning of services and resources for the orderplaced by the customer. At (5), TAS component 204 orchestrates theprovisioning of resources to support the subscribed services using theservices of SDI module 206. At (6) TAS module 204 provides informationrelated to the provisioned order received from SDI module 206 toservices module 202. In some embodiments, at (7), SDI module 206 mayalso use services provided by services module 202 to allocate andconfigure the resources needed to fulfill the customer's subscriptionorder.

At (8), services module 202 sends a notification to the customers onclient devices 224, 226 and 228 regarding the status of the order.

In certain embodiments, TAS module 204 functions as an orchestrationcomponent that manages business processes associated with each order andapplies business logic to determine whether an order should proceed toprovisioning. In one embodiment, upon receiving an order for a newsubscription, TAS module 204 sends a request to SDI module 206 toallocate resources and configure those resources needed to fulfill thesubscription order. SDI module 206 enables the allocation of resourcesfor the services ordered by the customer. SDI module 206 provides alevel of abstraction between the cloud services provided by cloudinfrastructure system 100 and the physical implementation layer that isused to provision the resources for providing the requested services.TAS module 204 may thus be isolated from implementation details such aswhether or not services and resources are actually provisioned on thefly or pre-provisioned and only allocated/assigned upon request.

In certain embodiments, a user may use store UI 210 to directly interactwith order management module 214 to perform billing and accountingrelated functions such as verifying the order and upon verification,booking the order. In some embodiments, instead of a customer placing anorder, at (9), the order may instead be placed by sales personnel 218 onbehalf of the customer such as a customer's service representative orsales representative. Sales personnel 218 may directly interact withorder management module 214 via a user interface (not shown in FIG. 2)provided by order management module 214 for placing orders or forproviding quotes for the customer. This, for example, may be done forlarge customers where the order may be placed by the customer's salesrepresentative through order management module 214. The salesrepresentative may set up the subscription on behalf of the customer.

EM module 208 is configured to monitor activities related to managingand tracking a customer's subscription in cloud infrastructure system100. EM module 208 collects usage statistics for the services in thesubscription order such as the amount of storage used, the amount datatransferred, the number of users, and the amount of system up time andsystem down time. At (10), a host operator personnel 220, who may be anemployee of a provider of cloud infrastructure system 100, may interactwith EM module 208 via an enterprise manager user interface (not shownin FIG. 2) to manage systems and resources on which services areprovisioned within cloud infrastructure system 100.

Identity management (IDM) module 200 is configured to provide identityservices such as access management and authorization services in cloudinfrastructure system 100. In one embodiment, IDM module 200 controlsinformation about customers who wish to utilize the services provided bycloud infrastructure system 100. Such information can includeinformation that authenticates the identities of such customers andinformation that describes which actions those customers are authorizedto perform relative to various system resources (e.g., files,directories, applications, communication ports, memory segments, etc.)IDM module 200 can also include the management of descriptiveinformation about each customer and about how and by whom thatdescriptive information can be accessed and modified.

In one embodiment, information managed by the identity management module200 can be partitioned to create separate identity domains. Informationbelonging to a particular identity domain can be isolated from all otheridentity domains. Also, an identity domain can be shared by multipleseparate tenants. Each such tenant can be a customer subscribing toservices in the cloud infrastructure system 100. In some embodiments, acustomer can have one or many identity domains, and each identity domainmay be associated with one or more subscriptions, each subscriptionhaving one or many services. For example, a single customer canrepresent a large entity and identity domains may be created fordivisions/departments within this large entity. EM module 208 and IDMmodule 200 may in turn interact with order management module 214 at (11)and (12) respectively to manage and track the customer's subscriptionsin cloud infrastructure system 100.

In one embodiment, at (13), support services may also be provided to thecustomer via a support UI 216. In one embodiment, support UI 216 enablessupport personnel to interact with order management module 214 via asupport backend system to perform support services at (14). Supportpersonnel in the cloud infrastructure system 100 as well as customerscan submit bug reports and check the status of these reports via supportUI 216.

Other interfaces, not shown in FIG. 2 may also be provided by cloudinfrastructure system 100. For example, an identity domain administratormay use a user interface to IDM module 200 to configure domain and useridentities. In addition, customers may log into a separate interface foreach service they wish to utilize. In certain embodiments, a customerwho wishes to subscribe to one or more services offered by cloudinfrastructure system 100 may also be assigned various roles andresponsibilities. In one embodiment, the different roles andresponsibilities that may be assigned for a customer may include that ofa buyer, an account administrator, a service administrator, an identitydomain administrator or a user who utilizes the services and resourcesoffered by cloud infrastructure system 100. The different roles andresponsibilities are described more fully in FIG. 4 below.

FIG. 3A depicts a simplified flowchart 300 depicting processing that maybe performed by the TAS module in the cloud infrastructure system, inaccordance with an embodiment of the present invention. The processingdepicted in FIG. 3A may be implemented in software (e.g., code,instructions, program) executed by one or more processors, hardware, orcombinations thereof. The software may be stored in memory (e.g., on amemory device, on a non-transitory computer-readable storage medium).The particular series of processing steps depicted in FIG. 3A is notintended to be limiting. Other sequences of steps may also be performedaccording to alternative embodiments. For example, alternativeembodiments of the present invention may perform the steps outlinedabove in a different order. Moreover, the individual steps illustratedin FIG. 3A may include multiple sub-steps that may be performed invarious sequences as appropriate to the individual step. Furthermore,additional steps may be added or removed depending on the particularapplications. One of ordinary skill in the art would recognize manyvariations, modifications, and alternatives. In one embodiment, theprocessing depicted in FIG. 3A may be performed by one or morecomponents in TAS component 204 as will be described in detail in FIG.3B.

At 302, a customer's subscription order is processed. The processing mayinclude validating the order, in one example. Validating the orderincludes ensuring that the customer has paid for the subscription andensuring that the customer does not already have subscriptions with thesame name or that the customer is not attempting to create multiplesubscriptions of the same type in the same identity domain forsubscription types for which this is disallowed (such as, in the case ofa CRM service). Processing may also include tracking the status of anorder for each order that is being processed by cloud infrastructuresystem 100.

At 304, a business process associated with the order is identified. Insome instances, multiple business processes may be identified for anorder. Each business process identifies a series of steps for processingvarious aspects of the order. As an example, a first business processmay identify one or more steps related to provisioning physicalresources for the order, a second business process may identify one ormore steps related to creating an identity domain along with customeridentities for the order, a third business process may identify one ormore steps for related to performing back office functions such ascreating a customer record for the user, performing accounting functionsrelated to the order, and the like. In certain embodiments, differentbusiness processes may also be identified for processing differentservices in an order. For example, different business process may beidentified to process a CRM service and a database service.

At 306, the business process identified for the order in 304 isexecuted. Executing the business process associated with the order mayinclude orchestrating the series of steps associated with the businessprocess identified in step 304. For example, executing a businessprocess related to provisioning physical resources for the order mayinclude sending a request to SDI module 206 to allocate resources andconfigure those resources needed to fulfill the subscription order.

At 308, a notification is sent to the customer regarding the status ofthe provisioned order. Additional description related to performingsteps 302, 304, 306 and 308 is provided in detail in FIG. 3B.

FIG. 3B depicts a simplified high level diagram of one or moresub-modules in the TAS module in the cloud infrastructure system, inaccordance with an embodiment of the present invention. In oneembodiment, the modules depicted in FIG. 3B perform the processingdescribed in steps 302-308 discussed in FIG. 3A. In the illustratedembodiment, TAS module 204 comprises an order processing module 310, abusiness process identifier 312, a business process executor 316, anoverage framework 322, a workflow identification module 324, and abundled subscription generator module 326. These modules may beimplemented in hardware, or software, or combinations thereof. Thevarious modules of the TAS module depicted in FIG. 3B are meant forillustrative purposes only and are not intended to limit the scope ofembodiments of the present invention. Alternative embodiments mayinclude more or fewer modules than those shown in FIG. 3B.

In one embodiment, order processing module 310 receives an order from acustomer from one or more input sources 321. For example, orderprocessing module 310 may directly receive an order via cloud UI 212 orstore UI 210, in one embodiment. Alternatively, order processing module310 may receive an order from order management module 214 or orderdatabase 224. Order processing module 310 then processes the order. Incertain embodiments, processing the order includes generating a customerrecord which includes information about the order such as a servicetype, a service level, a customer level, the type of resources, theamount of the resources to be allocated to the service instance and atime period during which the service is desired. As part of theprocessing, order processing module 310 also determines whether theorder is a valid order. This includes ensuring that the customer doesnot already have subscriptions with the same name or that the customeris not attempting to create multiple subscriptions of the same type inthe same identity domain for subscription types where this is disallowed(such as, in the case of a fusion CRM service).

Order processing module 310 may also perform additional processing onthe order. Processing may include tracking the status of an order foreach order that is being processed by cloud infrastructure system 100.In one embodiment, order processing module 310 may process each order toidentify a number of states pertaining to the order. In one example, thedifferent states of an order may be an initialized state, a provisionedstate, an active state, an administration required state, an errorstate, and the like. An initialized state refers to the state of a neworder; a provisioned state refers to the state of an order once theservices and resources for the order have been provisioned. An order isin an active state when the order has been processed by TAS module 204and a notification to that effect has been delivered to the customer. Anorder is in an administration required state when intervention by anadministrator is needed to resolve the issue. The order is in an errorstate when the order cannot be processed. In addition to maintaining theorder progress status, order processing module 310 also maintainsdetailed information about any failures encountered during processexecution. In other embodiments, and as will be discussed in detailbelow, the additional processing performed by order processing module310 may also include changing the service level for a service in thesubscription, changing the services included in the subscription,extending the time period of the subscription, and canceling thesubscription or specifying different service levels for different timeperiods in the subscription.

After an order has been processed by order processing module 310,business logic is applied to determine whether the order should proceedto provisioning. In one embodiment, as part of orchestrating the order,business process identifier 312 receives the processed order from orderprocessing module 310 and applies business logic to identify aparticular business process to use for the order being processed. In oneembodiment, business process identifier 312 may utilize informationstored in a service catalog 314 to determine the particular businessprocess to be used for the order. In one embodiment, and as discussed inFIG. 3A, multiple business processes may be identified for an order andeach business process identifies a series of steps for processingvarious aspects of the order. In another embodiment, and as discussedabove, different business processes may be defined for different typesof services, or combinations of services such as a CRM service or adatabase service. In one embodiment, service catalog 314 may storeinformation mapping an order to a particular type of business process.Business process identifier 312 may use this information to identify aspecific business process for the order being processed.

Once a business process has been identified, business process identifier312 communicates the particular business process to be executed tobusiness process executor 316. Business process executor 316 thenexecutes steps of the identified business process by operating inconjunction with one or more modules in the cloud infrastructure system100. In some embodiments, business process executor 316 acts as anorchestrator for performing the steps associated with a businessprocess. For example, the business process executor may interact withorder processing module 310 to execute steps in a business process thatidentifies workflows related to the order, determines the overage ofservices in the order or identifies service components related to theorder.

In one example, business process executor 316 interacts with SDI module206 to execute steps in a business process for allocating andprovisioning resources for services requested in the subscription order.In this example, for each step in the business process, business processexecutor 316 may send a request to SDI component 206 to allocateresources and configure resources needed to fulfill the particular step.SDI component 206 is responsible for the actual allocation of theresources. Once all the steps of the business processes of an order havebeen executed, business process executor 316 may send a notification tothe customer of the processed order by utilizing the services ofservices component 202. The notification may include sending an emailnotification to the customer with details of the processed order. Theemail notification may also include deployment information related tothe order to enable the customer to access the subscribed services.

In certain embodiments, TAS module 204 may provide one or more TASApplication Programming Interfaces (APIs) 318 that enable TAS module 204to interact with other modules in cloud infrastructure system 100 andfor other modules to interact with TAS module 204. For example, the TASAPIs may include a system provisioning API that interacts with SDImodule 206 via an asynchronous Simple Object Access Protocol (SOAP)based web services call to provision resources for the customer'ssubscription order. In one embodiment, TAS module 204 may also utilizethe system provisioning API to accomplish system and service instancecreation and deletion, switch a service instance to an increased servicelevel, and associate service instances. An example of this is theassociation of a Java service instance to a fusion applications serviceinstance to allow secure web service communications. The TAS APIs mayalso include a notification API that interacts with the services module202 to notify the customer of a processed order. In certain embodiments,the TAS module 204 also periodically propagates subscriptioninformation, outages, and notifications (e.g. planned downtime) toservices component 202.

In certain embodiments, TAS module 204 periodically receives usagestatistics for each of the provisioned services such as the amount ofstorage used, the amount data transferred, the number of users, and theamount of system up time and system down time from EM module 208.Overage framework 322 utilizes the usage statistics to determine whetherover use of a service has occurred, and if so, to determine how much tobill for the overage, and provides this information to order managementmodule 214.

In certain embodiments, TAS module 204 includes an order workflowidentification module 324 that is configured to identify one or moreworkflows associated with processing a customer's subscription order. Incertain embodiments, TAS module 204 may include a subscription ordergeneration framework 326 for generating subscription orders for acustomer when the customer places a subscription order for one or moreservices offered by the cloud infrastructure system 100. In oneembodiment, a subscription order includes one or more service componentsresponsible for providing the services requested by a customer in thesubscription order.

Additionally, TAS module 204 may also interact with one or moreadditional databases such as a Tenant Information System (TIS) database320 to enable the provisioning of resources for one or more servicessubscribed by the customer while taking into consideration historicalinformation, if any, available for the customer. TIS database 320 mayinclude historical order information and historical usage informationpertaining to orders subscribed by the customer.

TAS module 204 may be deployed using different deployment models. Incertain embodiments, the deployment includes a central component thatinterfaces with one or more distributed components. The distributedcomponents may, for example, be deployed as various data centers andaccordingly may also be referred to as data center components. Thecentral component includes capabilities to process orders andco-ordinate services in cloud infrastructure system 100, while the datacenter components provide capabilities for provisioning and operatingthe runtime system that provides the resources for the subscribedservices.

FIG. 4 depicts an exemplary distributed deployment of the TAS module,according to an embodiment of the present invention. In the embodimentdepicted in FIG. 4, the distributed deployment of TAS module 204includes a TAS central component 400 and one or more TAS Data Centers(DCs) components 402, 404 and 406. These components may be implementedin hardware, or software, or combinations thereof.

In one embodiment, the responsibilities of TAS central component 400include, without limitation, to provide a centralized component forreceiving customer orders, performing order-related business operationssuch as creating a new subscription, changing the service level for aservice in the subscription, changing the services included in thesubscription, and extending the time period of the subscription, orcanceling the subscription. The responsibilities of TAS centralcomponent 400 may also include maintaining and serving subscription dataneeded by cloud infrastructure system 100 and interfacing with ordermanagement module 214, support UI 216, cloud UI 212 and store UI 210 tohandle all the back-office interactions.

In one embodiment, the responsibilities of TAS DCs 402, 404 and 406include, without limitation, performing runtime operations fororchestrating the provisioning the resources for one or more servicessubscribed by the customer. TAS DCs 402, 404 and 406 also includecapabilities to perform operations such as locking, unlocking, enabling,or disabling a subscription order, collecting metrics related to theorder, determining the status of the order, and sending notificationevents related to the order.

In an exemplary operation of the distributed TAS system shown in FIG. 4,TAS central component 400 initially receives an order from a customervia cloud UI 212, store UI 210, via order management system 214, or viaorder database 224. In one embodiment, the customer represents a buyerwho has financial information and the authority to order and/or change asubscription. In one embodiment, the order information includesinformation identifying the customer, the type of services that thecustomer wishes to subscribe to, and an account administrator who willbe responsible for handling the request. In certain embodiments, theaccount administrator may be nominated by the customer when the customerplaces an order for a subscription to one or more services offered bycloud infrastructure system 100. Based on the order information, the TAScentral component 400 identifies the data region of the world such asAmericas, EMEA, or Asia Pacific in which the order originates and theparticular TAS DCs (for e.g., 402, 404 or 406) that will be deployed forprovisioning the order. In one embodiment, the particular TAS DC (fore.g., from among DCs 402, 404 or 406) that will be deployed forprovisioning the order is determined based on the geographical dataregion in which the request originated.

TAS central component 400 then sends the order request to the particularTAS DC in which to provision services for the order request. In oneembodiment, TAS DCs 402, 404 or 406 identify a service administrator andan identity domain administrator responsible for processing the orderrequest at the particular TAS DC. The service administrator and theidentity administrator may be nominated by the account administratoridentified in the subscription order. TAS DCs 402, 404 or 406communicate with SDI module 204 to orchestrate the provisioning ofphysical resources for the order. SDI component 204 in respective TASDCs 402, 404 or 406 allocates resources and configures those resourcesneeded to fulfill the subscription order.

In certain embodiments, TAS DCs, 402, 404 or 406 identify an identitydomain associated with the subscription. SDI component 206 may providethe identity domain information to IDM component 200 (shown in FIG. 2)for identifying an existing identity domain or creating a new identitydomain. Once the order is provisioned by the SDI module at respectiveTAS DCs, 402, 404 or 406, TAS central component 400 may placeinformation regarding the provisioned resources in a support system, viasupport UI 216. Information may include, for example, displayingresource metrics related to the services and usage statistics of theservices.

Once in operation, at each data center, EM module 208 to periodicallycollects usage statistics for each of the provisioned servicesprovisioned at that data center, such as the amount of storage used, theamount data transferred, the number of users, and the amount of systemup time and system down time. These statistics are provided to the TASDC that is local to EM module 208 (i.e., at the same data center). In anembodiment, the TAS DCs may use the usage statistics to determinewhether overuse of a service has occurred, and if so, to determine howmuch to bill for the overage, and provide the billing information toorder management system 214.

FIG. 5 is a simplified block diagram illustrating the interactions ofthe SDI module with one or more modules in the cloud infrastructuresystem, in accordance with an embodiment of the present invention. Inone embodiment, SDI module 206 interacts with TAS module 204 toprovision resources for services in a subscription order received by TASmodule 204. In certain embodiments, one or more of the modulesillustrated in FIG. 5 may be modules within cloud infrastructure system100. In other embodiments, one or more of the modules that interact withSDI module 206 may be outside cloud infrastructure system 100. Inaddition, alternative embodiments may have more or less modules thanthose shown in FIG. 5. These modules may be implemented in hardware, orsoftware, or combinations thereof.

In one embodiment, the modules in SDI module 206 may include one or moremodules in SaaS platform 102 and PaaS platform 104 in cloudinfrastructure system 100. In order to perform provisioning of resourcesfor various services, SDI module 206 may interact with various othermodules, each customized to help with provisioning resources for aparticular type of service. For example, as illustrated in FIG. 5, SDImodule 206 may interact with a Java service provisioning control module500 to provision Java cloud services. In one embodiment, Java serviceprovisioning control component 500 may deploy a Java Cloud Service (JCS)assembly specified by SDI module 206 that includes a set of tasks to beperformed to provision Java cloud services. Infrastructure resources 106then determines the resources needed to provision the Java cloudservices.

As other examples, SDI module 206 may interact with one or more modulessuch as a Virtual Assembly Builder (VAB) module 502, an ApplicationExpress (APEX) deployer module 504, a Virtual Machine (VM) module 506,an IDM module 200, and a database machine module 118. VAB module 502includes capabilities to configure and provision complete multi-tierapplication environments. In one embodiment, VAB module 502 deploys aMiddleware (MW) service assembly specified by SDI module 206 toprovision a MW service in cloud infrastructure system 100 using theservices provided by VM module 506. APEX deployer module 504 includescapabilities to configure and provision database services. In oneembodiment, APEX deployer module 504 deploys a database service assemblyspecified by SDI module 206 to provision a database service in cloudinfrastructure system 100 using the resources provided by infrastructureresources 106. SDI module 206 interacts with IDM module 200 to provideidentity services such as access management across multiple applicationsin cloud infrastructure system 100.

FIG. 6 depicts a simplified high level diagram of sub-modules of the SDImodule according to an embodiment of the present invention. In theembodiment depicted in FIG. 6, SDI module 206 includes a SDI-WebServices (WS) module 600, an SDI request controller module 602, an SDItask manager module 604, an SDI monitoring module 606, an SDI dataaccess module 608, an SDI common library module 610, and an SDIconnector module 612. These modules may be implemented in hardware, orsoftware, or combinations thereof. SDI module 206 depicted in FIG. 6 andits various modules are meant for illustrative purposes only and are notintended to limit the scope of embodiments of the present invention.Alternative embodiments may have more or less modules than those shownin FIG. 6. These modules and their functions are described in detailbelow.

SDI-WS module 600 includes capabilities for receiving a step in thebusiness associated with an order from business process executor 316 ofTAS component 204. In one embodiment, SDI-WS module 600 parses each stepof the business process and converts the step into an internalrepresentation used by SDI module 206. In one embodiment, each step ofthe business process associated with the order arrives through a webservice processing layer (for example, via System Provisioning APIdiscussed in FIG. 3B) in the form of a SOAP request to SDI-WS module600.

SDI request controller module 602 is the internal request processingengine in SDI module 206 and includes capabilities for performingasynchronous request processing, concurrent request processing,concurrent task processing, fault tolerant and recovery and plug-insupport related to the order requests. In one embodiment, SDI requestcontroller module 602 accepts each step of the business processassociated with the order from SDI-WS module 600 and submits the step toSDI task manager module 604.

SDI task manager module 604 translates each step specified in thebusiness process into a series of tasks for provisioning the particularstep. Once the set of tasks for a specific step have been provisioned,SDI task manager module 604 responds to business process executor 316 inTAS module 204 with operation results that includes an order payloadwith details of the resources provisioned to fulfill the particularstep. SDI task manager module 604 repeats this process until all thesteps of the particular business process associated with the order arecomplete.

In certain embodiments, SDI task manager module 604 translates each stepspecified in the business process into a series of tasks by utilizingthe services of SDI connector module 612. SDI connector module 612includes one or more connectors for handling the deployment of tasksspecified by SDI task manager module 604 to provision one or moreservices related to the order request. In certain embodiments, one ormore of the connectors may handle tasks that are specific to aparticular service type while other connectors may handle tasks that arecommon across different service types. In one embodiment, SDI connectormodule 612 includes a set of connectors (wrapper APIs) that interfacewith one or more of the external modules (shown in FIG. 5) in cloudinfrastructure system 100 to provision the services and resourcesrelated to the order request. For example, Application Express (APEX)connector 614 interfaces with APEX deployer module 504 to provisiondatabase services. Web Center Connector 616 (WCC) interfaces with a webcenter module in cloud infrastructure system 100 to provision webservices. The web center module is a user engagement platform andincludes capabilities for delivering connectivity between people andinformation in cloud infrastructure system 100.

In certain embodiments, Middleware Applications (MA) connector 618interfaces with VAB module 502 in cloud infrastructure system 100 toprovision middleware application services. NUVIAQ connector 620interfaces with VAB module 502 to provision Java services. IDM connector622 interfaces with IDM module 200 to provide identity and accessmanagement for users subscribing to services and resources in cloudinfrastructure system 100. Virtual Assembly Builder (VAB) connector 624interfaces with VAB module 502 in cloud infrastructure system 100 toconfigure and provision complete multi-tier application environments.Plug-in connector 626 interfaces with EM module 208 to manage andmonitor the components in cloud infrastructure system 100. HTTP serverconnector 628 interfaces with one or more web servers in the PaaSplatform to provide connection services to users in cloud infrastructuresystem 100.

SDI monitoring module 606 in SDI module 206 provides an inboundinterface for receiving Java Management Extensions (JMX) requests. SDImonitoring module 606 also provides tools for managing and monitoringapplications, system objects and devices in cloud infrastructure system100. SDI-data access module 608 provides an inbound interface forreceiving Java Database Connectivity (JDBC) requests. SDI-data accessmodule 608 supports data access and provides object relational mapping,java transaction API services, data access objects, and connectionpooling in cloud infrastructure system 100. The SDI-common librarymodule 610 provides configuration support for the modules in SDI module206.

The embodiment of FIG. 6 discussed above describes modules in the SDImodule according to an embodiment of the present invention. FIG. 7Adepicts a simplified flowchart 700 depicting processing that may beperformed by the modules of the SDI module in the cloud infrastructuresystem, in accordance with an embodiment of the present invention. Theprocessing depicted in FIG. 7A may be implemented in software (e.g.,code, instructions, program) executed by one or more processors,hardware, or combinations thereof. The software may be stored in memory(e.g., on a memory device, on a non-transitory computer-readable storagemedium). The particular series of processing steps depicted in FIG. 7Ais not intended to be limiting. Other sequences of steps may also beperformed according to alternative embodiments. For example, alternativeembodiments of the present invention may perform the steps outlinedabove in a different order. Moreover, the individual steps illustratedin FIG. 7A may include multiple sub-steps that may be performed invarious sequences as appropriate to the individual step. Furthermore,additional steps may be added or removed depending on the particularapplications. One of ordinary skill in the art would recognize manyvariations, modifications, and alternatives. In one embodiment, theprocessing depicted in FIG. 7A may be performed by one or more modulesin the SDI module 206 discussed in detail in FIG. 6.

At 702, a business process associated with a subscription order isreceived. In one embodiment, SDI-WS module 600 in SDI module 206receives one or more steps in the business process associated with thesubscription order from business process executor 316. At 704, each stepin the business process is translated into a series of tasks forprovisioning resources for the subscription order. In one embodiment,SDI task manager module 604 in SDI module 206 translates each stepspecified in the business process into a series of tasks by utilizingthe services of SDI connector module 612. At 706, the subscription orderis provisioned based on the series of tasks. In one embodiment, and asdiscussed in FIG. 6, SDI connector module 612 includes one or moreconnectors for handling the deployment of tasks specified by SDI taskmanager module 604 to provision resources for the services in thesubscription order.

As described above with respect to FIG. 6, SDI task manager module 604translates each step specified in a business process into a series oftasks by utilizing the services of SDI connector module 612, which mayinclude one or more connectors for handling the deployment of tasksspecified by SDI task manager module 604 to provision one or moreservices related to the order request. One or more of the connectors mayhandle tasks that are specific to a particular service type while otherconnectors may handle tasks that are common across different servicetypes. In one embodiment, SDI connector module 612 includes a set ofconnectors (wrapper APIs) that interface with one or more of theexternal modules (shown in FIG. 5) in cloud infrastructure system 100 toprovision the services and resources related to the order request. Forexample, a NUVIAQ connector 620 interfaces with VAB module 502 toprovision Java services.

FIG. 7B depicts a simplified block diagram showing the high-levelarchitecture of a Nuviaq system 710 and its relationships with othercloud infrastructure components according to an embodiment of thepresent invention. It should be appreciated that Nuviaq system 710depicted in FIG. 7B may have other components than those depicted inFIG. 7B. Further, the embodiment shown in FIG. 7B is only one example ofa cloud infrastructure system that may incorporate an embodiment of theinvention. In some other embodiments, Nuviaq system 710 may have more orfewer components than shown in FIG. 7B, may combine two or morecomponents, or may have a different configuration or arrangement ofcomponents.

In certain embodiments, Nuviaq system 710 may be configured to provide aruntime engine for orchestrating PaaS operations. Nuviaq system 710 mayprovide a web service API to facilitate integration with other productsand services. Nuviaq system 710 also provides support for complexworkflows in system provisioning, application deployment and associatedlifecycle operations and integrates with management and monitoringsolutions.

In the embodiment depicted in FIG. 7B, Nuviaq system 710 comprises aNuviaq proxy 712, a Nuviaq manager 714, and a Nuviaq database 716. Incertain embodiments, Nuviaq manager 714 provides an entry point intoNuviaq system 710, providing secure access to PaaS operations via theweb service API. Internally, it tracks system state in the database andcontrols job execution on the workflow engine. In a public cloud, Nuviaqmanager 714 may be accessed by the Tenant Provisioning system (SDI 206)and the Tenant Console, to drive provisioning and deployment operationsrespectively.

In one embodiment, Nuviaq manager 714 executes jobs asynchronously viaan internal workflow engine. A job may be a sequence of actions specificto a given PaaS workflow. Actions may be performed in order, withfailure in any step resulting in failure of the overall job. Manyworkflow actions delegate to external systems relevant to the workflow,such as the EM command line interface (cli). In one implementation,Nuviaq manager 714 application may be hosted in a 2-node WebLogiccluster with associated HTTP server (e.g., Oracle HTTP Server or OHS)instance, running inside a firewall.

In certain embodiments, Nuviaq proxy 712 is the public access point tothe Nuviaq API. In one embodiment, only Public API may be exposed here.Requests received by proxy 712 may be forwarded to Nuviaq manager 714.In one embodiment, Nuviaq proxy 712 runs outside the firewall, whereasmanager 714 runs within the firewall. In one implementation, Nuviaqproxy 712 application runs on a WebLogic cluster running outside thefirewall.

In certain embodiments, Nuviaq database 716 tracks various domainentities such as, without limitation, platform instance, deploymentplan, application, WebLogic domain, jobs, alerts, and the like. Primarykeys may be aligned with the Service Database where appropriate.

In one embodiment, Platform Instance 718 may contain all resourcesrequired for a WebLogic service for a given tenant.

Nuviaq system 710 may rely on additional systems of cloud infrastructuresystem 100 to carry out the workflows used the WebLogic cloud service.These dependencies may include dependencies on SDI 206, IDM 200, a virusscan system, a service database, CRM instances, and the like. Forexample, Nuviaq system 710 may depend upon functions performed by anAssembly Deployer in SDI 206. In one embodiment, the Assembly Deployeris a system to manage interactions with OVAB (Oracle Virtual AssemblyBuilder) and OVM (Oracle Virtual Machine). Capabilities of the AssemblyDeployer used by Nuviaq system 710 may include, without limitation,functions for deploying an assembly, un-deploying an assembly,describing assembly deployment, scaling appliance, and the like. In oneimplementation, Nuviaq system 710 accesses the Assembly Deployer via aweb service API.

In certain embodiments, security policies may require certain artifactsto be scanned for viruses before being deployed to an application. Cloudinfrastructure system 100 may provide a virus scan system for thispurpose that provides scanning as a service for multiple components ofthe public cloud.

In certain embodiments, a public cloud infrastructure may maintain aService Database containing information about tenants (e.g., customers)and their service subscriptions. Nuviaq workflows may access to thisdata in order to properly configure a WebLogic service as a client toother services that the tenant also subscribes to.

Nuviaq system 710 may depend on IDM 200 for its security integration. Incertain embodiments, Java Service instances can be associated with a CRMinstance. The association allows user applications deployed to theirJava Service instance to access a CRM instance though Web Service calls.

Various entities may use services provided by Nuviaq system 710. Theseclients of Nuviaq system 710 may include: a Tenant Console, which is anmanagement server (e.g., Oracle Management Server) based user interfacethat customers may access to manage their applications on their platforminstances; several IDEs such as Oracle IDEs (JDeveloper, NetBeans, andOEPE) have been extended to offer access to application lifecyclemanagement operations; one or more Command Line Interfaces (CLIs) thatare available to access lifecycle operations on the platform instances.

Provisioning use case for Nuviaq system 710—A Provision PlatformInstance use case is realized via the Create Platform Instance operationof the Nuviaq API. In the context of cloud infrastructure system 100, aservice instance with respect to the Nuviaq system corresponds to aNuviaq platform instance. A platform instance is assigned a uniqueidentifier is used on all subsequent operations related to thisinstance. A Platform Deployment descriptor provided to the CreatePlatform Instance action allows for properties to be set that modify theconfiguration of the platform instance to meet the subscriptionrequirements of the tenant. These properties may include for example:

Property #1: oracle.cloud.service.weblogic.size

-   -   Values: BASIC, STANDARD, ENTERPRISE    -   Description: Specifies the subscription type. This impacts the        number of servers, database limits and quality of service        settings.        Property #2: oracle.cloud.service.weblogic.trial    -   Values: TRUE, FALSE    -   Description: Indicates whether or not this is a trial        subscription.        Property #3: oracle.cloud.service.weblogic.crm    -   Values: CRM Service ID    -   Description: Identifies a CRM service to be associated with this        WebLogic service instance.

FIG. 7C depicts an example sequence diagram illustrating steps of aprovisioning process using a Nuviaq system according to an embodiment ofthe present invention. The sequence diagram depicted in FIG. 7C is onlyan example and is not intended to be limiting.

Install/Update Application use case—The Install Application operationdeploys an application to a running WebLogic Server after validatingthat the application archive meets the security requirements of thePublic Cloud. In one embodiment, the Application Deployment descriptorprovided to the Install Application action allows for properties to beset that modify the configuration of the application to meet thesubscription requirements of the tenant. These properties may includefor example:

Property: oracle.cloud.service.weblogic.state

Values: RUNNING, STOPPED

Description: Specifies the initial state of the application afterdeployment.

FIG. 7D depicts an example sequence diagram illustrating steps of adeployment process using a Nuviaq system according to an embodiment ofthe present invention. The sequence diagram depicted in FIG. 7D is onlyan example and is not intended to be limiting.

Referring back to FIG. 2, in certain embodiments, TAS 204 and SDI 206working in cooperation are responsible for provisioning resources forone or more services ordered by a customer from a set of servicesoffered by cloud infrastructure system 100. For example, in oneembodiment, for provisioning a database service, the automatedprovisioning flow may be as follows for a paid subscription:

(1) Customer places an order for a paid subscription to a service viaStore UI 210.

(2) TAS 204 receives the subscription order.

(3) When services are available TAS 204 initiates provisioning by usingthe services of SDI 206. TAS 204 may perform business processorchestration, which will execute the relevant business process tocomplete the provisioning aspect of the order. In one embodiment, TAS204 may use a BPEL (Business Process Execution Language) Process Managerto orchestrate the steps involved in the provisioning and handle thelifecycle operations.(4) In one embodiment, to provision a database service, SDI 206 may callPLSQL APIs in the CLOUD_UI to associate a schema for the requestingcustomer.(5) After successful association of a schema to the customer, SDIsignals TAS and TAS send a notification to the customer that thedatabase service is now available for use by the customer.(6) The customer may log into cloud infrastructure system 100 (e.g.,using an URAL such as cloud.oracle.com) and activate the service.

In some embodiments, a customer may also be allowed to subscribe to aservice on a trial basis. For example, such a trial order may bereceived via cloud UI 212 (e.g., using cloud.oracle.com).

In certain embodiments, cloud infrastructure system 100 enablesunderlying hardware and service instances to be shared between customersor tenants. For example, the database service may be provisioned asshown in FIG. 7E in one embodiment. FIG. 7E depicts multiple Exadatacompute nodes 730 and 732, each providing a database instanceprovisioned for the database service. For example, compute node 730provides a database instance 734 for a database service. Each Exadatacompute node may have multiple database instances.

In certain embodiments, each database instance can comprise multipleschemas and the schemas may be associated with different customers ortenants. For example, in FIG. 7E, database instance 734 provides twoschemas 736 and 738, each with its own tables. Schema 736 may beassociated with a first customer or tenant subscribing to a databaseservice and schema 738 may be associated with a second customer ortenant subscribing to the database service. Each tenant gets acompletely isolated schema. Each schema acts like a container that canmanage database objects including tables, views, stored procedures,triggers, etc. for the associated tenant. Each schema may have onededicated tablespace, with each tablespace having one data file.

In this manner, a single database instance can provide database servicesto multiple tenants. This not only enables sharing of underlyinghardware resources but also enables sharing of service instance betweentenants.

In certain embodiments, such a multi-tenancy system is facilitated byIDM 200, which beneficially enables multiple separate customers, eachhaving their own separate identity domains, to use hardware and softwarethat is shared in the cloud. Consequently, there is no need for eachcustomer to have its own dedicated hardware or software resources, andin some cases resources that are not being used by some customers at aparticular moment can be used by other customers, thereby preventingthose resources from being wasted. For example, as depicted in FIG. 7E,a database instance can service multiple customers each with theirrespective identity domains. Although each such database serviceinstance can be a separate abstraction or view of a single physicalmulti-tenant database system that is shared among the many separateidentity domains, each such database service instance can have aseparate and potentially different schema than each other databaseservice instance has. Thus, the multi-tenant database system can storemappings between customer-specified database schemas and the identitydomains to which those database schemas pertain. The multi-tenantdatabase system can cause the database service instance for a particularidentity domain to use the schema that is mapped to that particularidentity domain.

The multi-tenancy can also be extended to other services such as theJava Service. For example, multiple customers can have a JAVA serviceinstance placed within their respective identity domains. Each suchidentity domain can have a JAVA virtual machine, which can be viewed asbeing a virtual “slice” of hardware. In one embodiment, a job-monitoringservice (e.g., Hudson) can be combined with a JAVA enterprise editionplatform (e.g., Oracle WebLogic) in the cloud to enable each separateidentity domain to have its own separate virtual “slice” of the JAVAenterprise edition platform. Such a job-monitoring service can, forexample, monitor the execution of repeated jobs, such as building asoftware project or jobs run by an operating system's time-based jobscheduler. Such repeated jobs can include the continuous building and/ortesting of software projects. Additionally or alternatively, suchrepeated jobs can include the monitoring of executions of operatingsystem-run jobs that are executed on machines that are remote from themachine on which the job-monitoring service executes.

TAS Order Flow Control

According to some embodiments, TAS can implement a flow control (e.g.,throttling, prioritizing) of customer orders. For example, TAS may use aTAS order flow control module 805 to accommodate scenarios wherein theavailable physical resources for hosted services in the cloud datacenter may not meet the incoming demand for service subscription orders.

FIG. 8 depicts a simplified high level diagram of the components in theTAS module 204 that may incorporate aTAS order flow control module 805,according to some embodiments. In one embodiment, the componentsdepicted in FIG. 8 perform the processing described in steps 302-308discussed in FIG. 3A. These components may be implemented in hardware,or software, or combinations thereof. The various components of the TASmodule depicted in FIG. 8 are meant for illustrative purposes only andare not intended to limit the scope of embodiments of the presentinvention.

In one embodiment, TAS order flow control module 805 receives an orderfrom a customer from one or more input sources 321. For example, orderprocessing module 310 may directly receive an order via cloud UI 212 orstore UI 210, in one embodiment. Alternatively, order processing module310 may receive an order from order management module 214 or orderdatabase 224. TAS order flow control module 805 controls the TAS orderflow before sending it to order processing module 310 for processing.

In one embodiment, TAS order flow control module 805 may control the TASorder flow based on set of rules 810 (e.g., flow control rules). In someinstances the set of rules 810 can be preset by an administrator.Additionally, TAS order flow control module 805 may control the TASorder flow based on system-related information 815. In some instances,system-related information 815 can be received from EM module 208, SDImodule 206, and/or TAS module 204.

In certain embodiments, TAS order flow control module 805 can receivesystem-related information 815 from EM module 208. For example, usagestatistics for each of the provisioned services such as the amount ofstorage used, the amount data transferred, the number of users, and theamount of system up time and system down time can be received from EMmodule 208.

TAS order flow control module 805 can be a customizable model forhandling of service capacity by throttling the provisioning of ordersbased on the system-related information 815. For example, SDI may have afinite set of resources and can receive more requests than SDI cansatisfy. Therefore, in some instances, the orders can be throttled dueto a lack of physical resources to satisfy the received orders.

In one example, system-related information 815 can include the differentstates of an order, which may be an initialized state, a provisionedstate, an active state, an administration required state or an errorstate. In another example, system-related information 815 can includethe amount of resources currently used for provisioning each service. Insome instances, system-related information 815 can include informationabout the service level for a service in the subscription, servicesincluded in the subscription, time period of the subscription.

Additionally, the TAS order flow control module 805 can control the flowof orders based on set of rules 810. For example, the set of rules 810can be based on, but not limited to, available capacity of eachresource, set threshold limits for each resource, priority class ofcustomer, and order placement in the queue for physical provisioning.

Furthermore, the throttling of orders by TAS can be completelytransparent to customers. Additionally, in some instances, a customermay be informed of delays, rejections, estimated time of completion fortheir submitted orders, but the order may not be rejected at orderplacement time.

For instance, a customer may order a service using the Cloud UI. Thecloud UI can send the order request to TAS for processing. When TASpicks up the order for processing, TAS module 204 can apply throttlemechanisms using the TAS order flow control module 805. As a result, therequest order may go through the order flow control module 805 beforebeing sent to SDI module 206 for provisioning.

For example, TAS can receive capacity information from the SDI module206, EM module 208, and/or TAS module 204 via a system-relatedinformation 815 notification. In some instances, If SDI run out ofcapacity or surpass the set threshold capacity, then the TAS ordercontrol module 805 can step in to control the flow of orders (e.g.,pause some orders). Furthermore, the orders can automatically beprovisioned once the order flow control module is notified that capacityhas been added. Additionally, the TAS order flow control module canestimate future resources that are needed based on orders which are in apause state.

According to one embodiment, for every requested order received, TAS inreal time can contact SDI to determine current capacity. For example, bychecking the capacity before each order is processed, TAS can make surethat multiple orders are not processed when there is only one order leftbefore the threshold limit is reached.

In some instances, based on the set of rules 810 sent to the TAS orderflow control module 805, when the threshold limit has been reach, TAScan sent customer orders in a for a manual workflow approval, pause theorder, or reject the order.

The manual workflow approval process can use a human administrator(e.g., operator personnel 220) to individually approve each requestedorder. If the requested order is approved in the manual workflowprocess, then the requested order can be sent back to the queue forprocessing with a different priority class. For example, the approvedworkflow orders from normal customers can be the highest priority afterthe orders from the preferred customers (e.g., gold customers). Anapprove order from the workflow can have a higher priority than a normalorder because the order has been manually reviewed and approved.However, in some embodiments, a percentage of the resources are reservedfor preferred customers, therefore even an approved order may not beprocessed until all the orders from the preferred customers areprocessed.

According to some embodiments, TAS can have functionality for flowcontrol, which may include: auto-handling of over-capacity; preferredcustomer list; and order processing modes.

I. Auto-Handling of Over-Capacity

In order to accommodate over-capacity (e.g., where the demand forservice subscriptions is greater than available supply of physicalresources) the TAS flow control module 805 may implement certaincontrols.

FIG. 9 depicts a simplified flowchart depicting processing that may beperformed by the TAS flow control module 805 in the cloud infrastructuresystem, in accordance with an embodiment of the present invention. Theprocessing 900 depicted in FIG. 9 may be implemented in software (e.g.,code, instructions, program) executed by one or more processors,hardware, or combinations thereof. The software may be stored in memory(e.g., on a memory device, on a non-transitory computer-readable storagemedium). The particular series of processing steps depicted in FIG. 9 isnot intended to be limiting.

At 902, TAS flow control module 805 can receiving an order for a servicefrom a plurality of service provided by the cloud infrastructure system.For example, TAS flow control module 805 can receive a customer orderfrom the store UI 210, cloud UI 212, order management system 214, ororder database 214, as previously described in FIG. 8. In one example,after receiving an order, TAS flow control module 805 can send a statusrequest of available resources for the provisioning of the serviceassociated with a customer order. The status request can allow TAS flowcontrol module 805 to system-related information 815 for theprovisioning of the service associated with a customer order. Forexample, system-related information 815 can include an unavailability ofresources, a threshold limit met, or a percentage of resources used.

According to one embodiment, the status request (e.g., SDI provisioningcall) may return a percentage of resources used. In this instance, theTAS flow control module can set a threshold limit and adjust the orderflow accordingly once the threshold has been met. TAS can receive amatrix of available resources and threshold limits for each service.

At 904, TAS flow control module 805 can determine based upon a set ofrules and system-related information, whether the order is to beforwarded to order processing system 310 for provisioning one or moreresources for enabling the order.

At 906, based upon the determining at 904, TAS flow control module 805can forward the order for provisioning the one or more resources or notforwarding the order for provisioning. At 906, the TAS flow controlmodule 805 can throttle the provisioning of the service associated withthe customer order based on set of rules 810. In some instances the flowcontrol rules can be based on priority, time, or order processing mode.

FIG. 10 depicts a simplified high level diagram of the components in TASorder flow control module 805, according to some embodiments. In oneembodiment, the components depicted in FIG. 10 perform the processingdescribed in steps 902-906 discussed in FIG. 9. These components may beimplemented in hardware, or software, or combinations thereof. Thevarious components of TAS order flow control module 805 depicted in FIG.10 are meant for illustrative purposes only and are not intended tolimit the scope of embodiments of the present invention.

According to some embodiments, a customer order from order database 224can be sent to the queue 910 in the TAS order flow control module 805.From queue 910, an order selector 915 can prioritize the orders based onset of rules 810. Then, order selector 915 can sent the prioritizedorders to a throttling controller 920. Throttling controller 920 candetermine whether to send the order to: a pause queue 925; a marked asfailed queue 930; a workflow queue 935; or order processing system 310based on system-related information 815 and set of rules 810.

For example, when the SDI provisioning call returns an unavailableresource notification (e.g., RESOURCE_UNAVAILABLE error) for a givenservice, the TAS flow control module 805 can throttle the provisioningof the service. Throttling can include, but is not limited to: sendingthe order to pause queue 925; marking the order as failed and resubmitby sending the order to marked as failed queue 930; sending the order toworkflow queue 935; sending the order for processing by sending theorder to order processing system 310; or sending an email notificationto an administrator.

In the pause queue 925, throttling controller 920 can pause anysubsequent orders for the service that is over-capacity and resumeprocessing of the orders only when the available capacity is increased.This can be done so as to not burden administrators to manually resumeall orders failed due to unavailable capacity once capacity becomesavailable. Thus, TAS flow control module 805 may pause subsequent ordersbased on the over-capacity.

When an order is paused, throttling controller 920 can send a notice tothe customer informing the customer that the order is paused and will beprocessed soon. In some instances, the order flow control module cansend periodic notification with status updates for the order.

In the marked as failed queue 930, throttling controller 920 can markthe failed order (e.g., mark as resubmit-upon-available-capacity) andresubmit the order to the queue 910 with the same request-id to SDI oncecapacity becomes available. According to another embodiment, throttlingcontroller 920 may not resubmit the order. This action may be taken forall in-flight orders that run into the unavailable resource notification(e.g., RESOURCE_UNAVAILABLE error).

In the workflow queue 935, throttling controller 920 can send the orderto a workflow. The workflow can be a manual (e.g., human-based)approving process. The workflow can be used to filter out the ordersthat are less likely to be used. For example, a customer may place anorder without the intentions of using the service. In the workflow, anadministrator can manually approve each order that comes into theworkflow. For example, an administrator (e.g., project managers) can goto conferences and meet potential customers and collect informationabout the potential customer. Additionally, all orders from certaincountries may have to go through the workflow for approval. Once therequested order has been approved by the workflow, the order can beprocessed or the order can be sent back to the queue with a higherpriority level. Sometimes the approved workflow order can be sent backto the order flow control module because an approved workflow order maystill have a lower priority than a gold customer order.

According to another embodiment, throttling controller 920 can send theorder to order processing system 310 for provisioning the requestedservice. FIG. 3B describes the process for provisioning the requestedservice once the order is received by order processing system 310.

According to another embodiment, the TAS flow control module 805 cansend an email notification to an administrator to indicate that SDIreturned an unavailable resource notification (e.g., aRESOURCE_UNAVAILABLE exception), so that the administrator can procuremore hardware. According to another embodiment, SDI can send anotification directly to the administrator; therefore TAS may skip thisstep.

Furthermore, as TAS keeps track of any over-capacity status on aper-service basis (e.g., when SDI sends a RESOURCE_UNAVAILABLEexception), TAS may maintain this status separately for the common pooland for the preferred customer reserved pool. This may allow us tothrottle regular order requests independently of preferred customerorder requests.

FIG. 11 depicts a simplified flowchart depicting processing that may beperformed by TAS flow control module 805 in the cloud infrastructuresystem, in accordance with another embodiment of the present invention.The processing 950 depicted in FIG. 11 may be implemented in software(e.g., code, instructions, program) executed by one or more processors,hardware, or combinations thereof. The software may be stored in memory(e.g., on a memory device, on a non-transitory computer-readable storagemedium). The particular series of processing steps depicted in FIG. 11is not intended to be limiting.

At 955, TAS flow control module 805 can receive an order from inputsources 321. At 960, TAS flow control module 805 can queue order intoqueue 910. At 965, TAS flow control module 805 can select from queue 910based on set of priority rules (e.g., set of rules 810). At 970, TASflow control module 805 can apply throttling rules based on set ofthrottling rules (e.g., set of rules 810) and system related information815.

As previously described in FIG. 10, at 970, throttling controller 920can send the order to: a pause queue 925; a marked as failed queue 930;a workflow queue 935; or order processing system 310 based onsystem-related information 815 and set of rules 810.

At 975, TAS flow control module 805 can pause the order and at 980 TASflow control module 805 can notify the customer that the order ispaused. Alternatively, at 985, TAS flow control module 805 can add theorder to workflow queue for manual approval. Alternatively, at 990, TASflow control module 805 can reject the order and send the order to themark as failed queue. Alternatively, at 995, TAS flow control module 805can approve the order and sent the order to order processing system 310for processing and provisioning.

A. TAS Determining Increased Capacity

In some instances, The TAS order flow control module can poll SDI module206 or EM module 208 for an updated status request of the availableresources for provisioning of the service associated with the customerorder. The request information can be return as system-relatedinformation 815.

For example, once TAS gets an unavailable resource notification (e.g.,RESOURCE_UNAVAILABLE error) from SDI or EM for a given service, TAS mayresume processing orders for that service only after capacity isincreased for that service.

According to some embodiments, TAS may determine when the capacity hasbeen increased by: periodically calling SDI or EM to check availablecapacity; or the Ops team notifying TAS that the capacity has increased.For example, for each service that encountered the unavailable resourcenotification, a TAS background job may periodically call SDI to checkavailable capacity. Additionally, in some instances, the Ops team may beprovided a command line utility (e.g., tasctl command) to signal to TASthat capacity for a service has been increased.

B. Notification to the Customer

In some instances, TAS order flow control module 805 can notify thecustomer who requested the specific order with the status of theprovisioning of the service.

TAS may investigate a mechanism wherein it notifies the customer of anydelays in the provisioning (e.g., due to unavailable capacity, due toother reasons). For example, TAS can send the customer an initialnotification that the order was received. Subsequently, a background jobcan wake up periodically and send other notifications (e.g., “Pleasewait, the order is processing”). The timing of notification can beconfigurable (e.g., once every two days).

II. Prioritizing Customers and Threshold Mechanism

TAS order flow control module 805 can prioritize customers using orderselector 915 based on set of rules. Additionally, TAS order flow controlmodule 805 can use throttling controller 920 when implementing thethrottling of customer orders based on set of rules 810 andsystem-related information 815.

According to some embodiments, order selector 915 may have the abilityto fast track orders for preferred customers (e.g., gold customers).Order selector 915 prioritize customers by using an invitation codemechanism. Customers can get invitation codes using mechanism that canbe independent of the TAS overflow control module.

In some instances, invitations can have the following attributes:priority, custom trial period. Examples of priority can include: LOW,NORMAL, or HIGH. For example, gold customers can get HIGH priorityinvitation codes. Customers that have consistent large paid orders(e.g., over $100,000) can be gold customers.

Additionally, a customized trial period for prioritized customers canoverride the default trial period for a service. For example, goldcustomers may be given longer trial periods. Moreover, orders can beprioritized based on the status of a customer (e.g., a potential payingcustomer).

According to some embodiments, mechanisms can be implemented that once athreshold capacity has been passed; the remaining order can only be usedfor high priority customers (e.g., gold customers). For example, themaximum capacity can be 100 orders, and if current capacity is below aset threshold (e.g., current capacity is below 50 orders), then ordersare taken on a first come first serve basis. However, when the currentcapacity is above the set threshold (e.g., current capacity is over 50orders), and then the remaining orders are saved for provisioning ordersfrom preferred customers. The priority of customers and threshold can beset by an administrator.

In some instances, throttling controller 920 can control the flow oforders being provisioned. For example, once the threshold limits areset, (i.e. a threshold of 60 percent of the service capacity) then theremaining orders can go through the overflow control module. Forexample, the sixty-first order that's being processed by TAS will not gointo the regular flow. The sixty-first order goes through the overflowcontrol module, because the threshold limit has been reached. Theoverflow control module can then determine if the order should beprocessed based on set criteria (e.g., prioritized customer list)discussed herein.

Order processing may interpret the invitation code based on: an order ofpriority; service instances reserved for preferred customers; and acustom trial period.

For example, orders may be processed in order of priority (i.e. orderswith HIGH priority may be processed ahead of orders with LOW or NORMALpriority). Additionally, orders with the same priority may be processedin the time order in which they are submitted. The order flow controlmodule can sort based on priority and timing of requested order.

Moreover, TAS may pass the priority to SDI in order to allow SDI toallocate service instances from pools reserved for preferred customersin case of HIGH priority orders.

Furthermore, TAS may create the service subscription using the customtrial period, if specified as part of the invitation code. In someinstances, the TAS trial management code may not change to honor thesubscription period for each individual subscription.

Additionally, TAS can reject order from specific customers. For example,malicious denial of service can be one example of a rejected order. Insome instances, orders can be rejected based on the geographicallylocation of the order or a blacklist. In other instances, orders can berejected based other requirements (e.g., order is from a competitor).

For example, once TAS order flow control module 805 receives a customerorder, TAS order flow control module 805 determines the type of servicerequested. Then, TAS order flow control module 805 can contact thesystem provisioning for the service and request a resource capacitystatus. Based on the resource capacity status and predefined set ofrules (e.g., priority list), TAS order flow control module 805 can makea determination.

According to some embodiments, the processing of a requested order canbe based on timing (e.g., received timestamp associated with thecustomer order). For example, in addition to the other rules, the workflow control module can process orders on a first-come first-servedbasis for each order category.

III. Order Selector

In addition to managing the flow of orders when demand is greater thanavailable capacity, order selector 915 can provide the ability tocontrol which customer orders enter the order queue in the first place(e.g., based on the potential of the customer).

This functionality may be supported based on the order processing mode.The order processing mode can be set as a system-wide default and can beoptionally overridden on a per-service basis. The order processing modecan include: open plus optional invitation; manual approval; invitationonly; or invitation plus manual approval.

In the open plus optional invitation mode, (e.g.,OPEN_PLUS_OPTIONAL_INVITATION), all orders may be accepted. Customerscan optionally enter an invitation code. Orders may get processed on afirst-come, first-served basis unless the order has an invitation code,which may give their orders priority.

In the manual approval only mode (e.g., MANUAL_APPROVAL_ONLY), allorders may go through a human approval before being processed.

In the invitation only mode (e.g., INVITATION_ONLY), only orders withinvitation codes may be accepted. Additionally, orders may be processedin order of invitation code priority.

In the invitation plus manual approval mode, (e.g.,INVITATION_PLUS_MANUAL_APPROVAL), orders with invitation codes may beprocessed first in order of invitation code priority. Orders without aninvitation code may go through a human approval.

TAS order control module 805 may support a notion of invitation codes tocontrol which customers can request trial service subscriptions. TASorder control module 805 may support different methods of generatinginvitation codes.

For example, customers may be allowed to request invitations toparticipate in trial offerings for the service. TAS may provide an APIto request invitations. Invitation requests may go through a humanworkflow and may have to be approved by the Ops team and/oradministrators. Once approved, TAS may send a notification to thecustomer inviting them to participate along with their custom invitationcode. The customer can then use that invitation code to place trialorders. In processing the order, TAS may validate that the invitationcode matches the requestor's email address.

Alternatively, the Ops team and/or administrators can also generateinvitations for selected customers using a TAS API/UserInterface/Command-line interface. TAS may send a notification to thecustomer inviting them to participate along with their custom invitationcode. The customer can then use that invitation code to place trialorders. In processing the order, TAS may validate that the invitationcode matches the requestor's email address.

Regardless of the order processing mode used for a service, it can bepossible for the demand to exceed available capacity. Therefore, TAS canenforce the auto-handling of over-capacity independent of the supportfor the order processing mode. For instance, if there are only 100preferred customer reserved anonymous Java service instances in the pooland there are 200 preferred customer orders coming in, the TAS modulemay automatically enforce the over-capacity logic.

Additionally, TAS may provide command-line tools or user interface toallow Ops developer and administrators to: list all orders pendingapproval (e.g., by leveraging the BPEL Human Workflow); change priorityof a specific order (e.g., to mark some customer as a preferredcustomer); and pre-emptively terminate a subscription by changing thesubscription dates.

IV. HTTP Routing Layer

According to some embodiments, the cloud system can rely on the routingof HTTP traffic from published endpoints (e.g., URL) to individualservice instances that are actually serving content for the variouscustomer-facing services. This routing layer can be based on a common,shared set of HTTP server (e.g., Oracle's OHS) instances that areconfigured as reverse proxies to route traffic from the client to theappropriate service instance.

The HTTP routing layer can consist of multiple pools of HTTP serverswith one pool per service type. This can provide isolation at theservice layer from outages as well as configuration isolation on aservice-by-service basis.

Furthermore, TAS order control module 805 can use the routing scheme tomap services to instances can vary depending on service type, but eachrouting scheme determines the tenant and service instance to which arequest is intended and then routes the traffic to the appropriateservice instance. This mapping is based on information that is providedto the routing layer from SDI. For certain services, the routing schemecan be configured using the HTTP server. For others, an HTTP servermodule performs the tenant to service instance mapping.

Additionally, the HTTP routing layer can provide a simple throttlingmechanism to ensure that users are not trying to circumvent their userlimits by making large number of service calls to do something likeprovide an alternate user interface to their Fusion Applicationsinstance. The throttling algorithm limits the number of concurrent userrequests that a given tenant can have. An HTTP server module (e.g.,similar to mod_qos1) can do the throttling. Mod_qos1 is a loadablewebserver module that limits concurrent connections to the webserver.

FIG. 12 is a simplified block diagram of a computing system 1000 thatmay be used in accordance with embodiments of the present invention. Forexample, cloud infrastructure system 100 may comprise one or morecomputing devices. System 1000 depicted in FIG. 12 may be an example ofone such computing device. Computer system 1000 is shown comprisinghardware elements that may be electrically coupled via a bus 1024. Thecomponents may include one or more processing units 1002, an inputsubsystem 1004, an output subsystem 1006, storage devices 1008, acomputer-readable storage media reader 1012 connected to acomputer-readable storage medium 1010, a communication subsystem 1014, aprocessing acceleration subsystem 1016, and working memory 1018.

Bus subsystem 1024 provides a mechanism for letting the variouscomponents and subsystems of computer system 1000 communicate with eachother as intended. Although bus subsystem 1024 is shown schematically asa single bus, alternative embodiments of the bus subsystem may utilizemultiple busses.

Input subsystem 1004 may include one or more input devices such as amouse, a keyboard, a pointing device, a touchpad, etc. In general, inputsubsystem 1004 may include any device or mechanism for inputtinginformation to computer system 1000.

Output subsystem 1006 may include one or more output devices foroutputting information from computer system 1000. Examples of outputdevices include without limitation a display device, a printer, aprojection device, etc. In general, output subsystem 1006 may includeany device or mechanism for outputting information from computer system1000.

Processing unit(s) 1002 can include one or more processors, one or morecores of processors, combinations thereof, and the like. In someembodiments, processing unit(s) 1002 can include a general purposeprimary processor as well as one or more special purpose co-processorssuch as graphics processors, digital signal processors, or the like. Insome embodiments, some or all processing units 1002 can be implementedusing customized circuits, such as application specific integratedcircuits (ASICs) or field programmable gate arrays (FPGAs). In someembodiments, such integrated circuits execute instructions that arestored on the circuit itself. In other embodiments, processing unit(s)1002 can execute instructions stored in working memory 1018 or onstorage devices 1008. In various embodiments, processing units 1002 canexecute a variety of programs or code instructions and can maintainmultiple concurrently executing programs or processes. At any giventime, some or all of the program code to be executed can be resident insystem working memory 1018, storage devices 1008, and/or oncomputer-readable storage media 1010. Through suitable programming,processing units 1002 can provide various functionalities describedabove for performing event stream-related processing. In someembodiments, computer system 1000 may also include a processingacceleration unit 1016, which can include a digital signal processor(DSP), a special-purpose processor, and/or the like.

Storage device(s) 1008 may include memory devices such as disk drives,optical storage devices, and solid-state storage devices such as arandom access memory (RAM) and/or a read-only memory (ROM), which can beprogrammable, flash-updateable and/or the like. Software (programs, codemodules, instructions), which when executed by processing unit(s) 1002to provide the functionality described above, may be stored on storagedevices 1008. Storage devices 1008 may also provide a repository forstoring data used in accordance with embodiments of the presentinvention.

Computer-readable storage media reader 1012 can further be connected toa computer-readable storage medium 1010, together (and, optionally, incombination with storage device(s) 1008) comprehensively representingremote, local, fixed, and/or removable memory storage devices plusstorage media for temporarily and/or more permanently containingcomputer-readable information.

Communications subsystem 1014 may permit data to be exchanged withnetwork and/or any other computers. Communication subsystem 1014 servesas an interface for receiving data from and transmitting data to othersystems from computer system 1000. The communication may be providedusing wired or wireless protocols. For example, communication subsystem1014 may enable computer 1000 to connect to a client device via theInternet. Communication subsystem 1014 may comprise a modem, a networkcard (wireless or wired), an infra-red communication device, a GPSreceiver, etc.

Working memory subsystem 1018 may include a number of memories includinga main random access memory (RAM) for storage of instructions and dataduring program execution and a read only memory (ROM) in which fixedinstructions are stored. Software elements such as an operating system1020 and/or other code 1022, such as an application program (which maybe a client application, Web browser, mid-tier application, RDBMS,etc.), may be stored in working memory 1018. In an exemplary embodiment,working memory 1018 may include executable code and associated datastructures (such as caches) used for processing events and enablingvariable duration windows processing as described above.

It should be appreciated that alternative embodiments of computer system1000 may have more or less components with numerous variations from thatdescribed above. For example, customized hardware might also be usedand/or particular elements might be implemented in hardware, software(including portable software, such as applets), or both. Further,connection to other computing devices such as network input/outputdevices may be employed.

Although specific embodiments of the invention have been described,various modifications, alterations, alternative constructions, andequivalents are also encompassed within the scope of the invention.Embodiments of the present invention are not restricted to operationwithin certain specific data processing environments, but are free tooperate within a plurality of data processing environments.Additionally, although embodiments of the present invention have beendescribed using a particular series of transactions and steps, it shouldbe apparent to those skilled in the art that the scope of the presentinvention is not limited to the described series of transactions andsteps.

Further, while embodiments of the present invention have been describedusing a particular combination of hardware and software, it should berecognized that other combinations of hardware and software are alsowithin the scope of the present invention. Embodiments of the presentinvention may be implemented only in hardware, or only in software, orusing combinations thereof. The various processes described herein canbe implemented on the same processor or different processors in anycombination. Accordingly, where components or modules are described asbeing configured to perform certain operations, such configuration canbe accomplished, e.g., by designing electronic circuits to perform theoperation, by programming programmable electronic circuits (such asmicroprocessors) to perform the operation, or any combination thereof.Processes can communicate using a variety of techniques including butnot limited to conventional techniques for interprocess communication,and different pairs of processes may use different techniques, or thesame pair of processes may use different techniques at different times.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that additions, subtractions, deletions, and other modificationsand changes may be made thereunto without departing from the broaderspirit and scope as set forth in the claims. Thus, although specificinvention embodiments have been described, these are not intended to belimiting. Various modifications and equivalents are within the scope ofthe following claims.

What is claimed is:
 1. A method comprising: storing, by a computerresource infrastructure system, order information about an order for aservice from a plurality of services provided by the computer resourceinfrastructure system, the computer resource infrastructure systemcomprising one or more computing devices; determining, by a computingdevice from the one or more computing devices, based upon an orderprocessing rule and system-related information, whether a resourcecapacity of the computer infrastructure system meets a thresholdresource capacity, wherein the order processing rule includes thethreshold resource capacity, and wherein the system-related informationindicates the resource capacity; upon determining that the resourcecapacity does not meet the threshold resource capacity, sending arequest to provision one or more resources to enable the service for theorder; upon determining that the resource capacity meets the thresholdresource capacity: delaying the request from being sent, whereindelaying the request includes storing the order information within aqueue for future order processing, and wherein the queue stores orderinformation about one or more orders that are delayed; detecting anupdated resource capacity of the computer resource infrastructure systemafter delaying the request, wherein the updated resource capacitycorresponds to a change in the resource capacity; removing the orderinformation from the queue based on determining that the updatedresource capacity does not meet the threshold resource capacity; andupon removing the order information form the queue, sending the requestto provision the one or more resources to enable the service for theorder.
 2. The method of claim 1, wherein the order processing ruleincludes a service-specific threshold limit for available resourcesspecific to a type of service and wherein determining whether theresource capacity meets the threshold resource capacity includesdetermining whether the resource capacity meets the service-specificthreshold limit.
 3. The method of claim 2, further comprising:determining that the order is for a preferred customer according to acustomer priority list; and upon determining that the resource capacitymeets the threshold resource capacity, determining whether to delay therequest based on the customer priority list.
 4. The method of claim 1,wherein the determining comprises: determining that the order should bemarked as failed and resubmitted; and preventing the request from beingsent to provision the one or more resources to enable the service forthe order.
 5. The method of claim 1, wherein the determining comprises:determining that the order should be sent to workflow for a manualapproval by the administrator; and delaying the request from being sentto provision the one or more resources to enable the service for theorder until the order is manually approved by an administrator.
 6. Themethod of claim 1, further comprising: sending a notification toincrease available resources in the computer resource infrastructuresystem based on determining that the resource capacity meets thethreshold resource capacity.
 7. The method of claim 1, wherein the orderprocessing rule is based on a timestamp associated with the order. 8.The method of claim 1, wherein the set of rules further include rulesbased on an order processing mode, and wherein the order processing modeincludes: open plus optional invitation; manual approval; invitationonly; or invitation plus manual approval.
 9. The method of claim 1,wherein the system-related information includes: usage statistics foreach service of the plurality of services, an amount of storage used foreach service, an amount of data transferred for each service, a numberof users, an amount of system up time, or an amount of system down time.10. A system comprising: one or more computing devices configurable tooffer a set of services; memory configurable to store an order for aservice from the set of services provided by the computer resourceinfrastructure system; and wherein a computing device from the one ormore computing devices is configured to: determine based upon an orderprocessing rule and system-related information, whether a resourcecapacity of the computer resource infrastructure system meets athreshold resource capacity, wherein the order processing rule includesthe threshold resource capacity, and wherein the system-relatedinformation indicates the resource capacity; upon determining that theresource capacity does not meet the threshold resource capacity, send arequest to provision one or more resources to enable the service for theorder; upon determining that the resource capacity meets the thresholdresource capacity: delay the request from being sent, wherein delayingthe request includes storing the order information within a queue forfuture order processing, and wherein the queue stores order informationabout one or more orders that are delayed; detect an updated resourcecapacity of the computer resource infrastructure system after delayingthe request, wherein the updated resource capacity corresponds to achange in the resource capacity; remove the order information from thequeue based on determining that the updated resource capacity does notmeet the threshold resource capacity; and upon removing the orderinformation form the queue, send the request to provision the one ormore resources to enable the service for the order.
 11. The system ofclaim 10, wherein the order processing rule includes a service-specificthreshold limit for available resources specific to a type of serviceand wherein determining whether the resource capacity meets thethreshold resource capacity includes determining whether the resourcecapacity meets the service-specific threshold limit.
 12. The system ofclaim 11, wherein the computing device is further configured to:determine that the order is for a preferred customer according to acustomer priority list; and upon determining that the resource capacitymeets the threshold resource capacity, determine whether to delay therequest based on the customer priority list.
 13. The system of claim 10,wherein the computing device is further configured to: determine thatthe order should be marked as failed and resubmitted; and prevent therequest from being sent to provision the one or more resources to enablethe service for the order.
 14. The system of claim 10, wherein thecomputing device is further configured to: determine that the ordershould be sent to workflow for a manual approval by the administrator;and delay the request from being sent to provision the one or moreresources to enable the service for the order until the order ismanually approved by an administrator.
 15. The system of claim 10,wherein the computing device is further configured to: send anotification to increase available resources based on determining thatthe resource capacity meets the threshold resource capacity.
 16. Thesystem of claim 10, wherein the order processing rule is based on atimestamp associated with the order.
 17. The system of claim 10, whereinthe set of rules further include rules based on an order processingmode, and wherein the order processing mode includes: open plus optionalinvitation; manual approval; invitation only; or invitation plus manualapproval.
 18. The system of claim 10, wherein the system-relatedinformation includes: usage statistics for each service of the set ofservices, an amount of storage used for each service, an amount of datatransferred for each service, a number of users, an amount of system uptime, or an amount of system down time.
 19. One or more non-transitorycomputer-readable media storing computer executable instructions for acomputer resource infrastructure system configured to offer a set ofservices that, when executed, cause one or more computing devices in thecomputer resource infrastructure system to: store an order for a servicefrom the set of services provided by the computer resourceinfrastructure system; determine based upon an order processing rule andsystem-related information, whether a resource capacity of the computerresource infrastructure system meets a threshold resource capacity,wherein the order processing rule includes the threshold resourcecapacity, and wherein the system-related information indicates theresource capacity; upon determining that the resource capacity does notmeet the threshold resource capacity, sending a request to provision oneor more resources to enable the service for the order; upon determiningthat the resource capacity meets the threshold resource capacity: delaythe request from being sent, wherein delaying the request includesstoring the order information within a queue for future orderprocessing, and wherein the queue stores order information about one ormore orders that are delayed; detect an updated resource capacity of thecomputer resource infrastructure system after delaying the request,wherein the updated resource capacity corresponds to a change in theresource capacity; remove the order information from the queue based ondetermining that the updated resource capacity does not meet thethreshold resource capacity; and upon removing the order informationform the queue, send the request to provision the one or more resourcesto enable the service for the order.
 20. The one or more non-transitorycomputer-readable media of claim 19, wherein the order processing ruleincludes a service-specific threshold limit for available resourcesspecific to a type of service and wherein determining whether theresource capacity meets the threshold resource capacity includesdetermining whether the resource capacity meets the service-specificthreshold limit.
 21. The one or more non-transitory computer-readablemedia of claim 20, wherein the instructions, when executed, furthercause the one or more computing devices to: determine that the order isfor a preferred customer according to a customer priority list; and upondetermining that the resource capacity meets the threshold resourcecapacity, determine whether to delay the request based on the customerpriority list.
 22. The one or more non-transitory computer-readablemedia of claim 19, wherein the instructions, when executed, furthercause the one or more computing devices to: determine that the ordershould be marked as failed and resubmitted; and prevent the request frombeing sent to provision the one or more resources to enable the servicefor the order.
 23. The one or more non-transitory computer-readablemedia of claim 19, wherein the instructions, when executed, furthercause the one or more computing devices to: determine that the ordershould be sent to workflow for a manual approval by the administrator;and delay the request from being sent to provision the one or moreresources to enable the service for the order until the order ismanually approved by an administrator.
 24. The one or morenon-transitory computer-readable media of claim 19, wherein theinstructions, when executed, further cause the one or more computingdevices to: send a notification to increase available resources based ondetermining that the resource capacity meets the threshold resourcecapacity.
 25. The one or more non-transitory computer-readable media ofclaim 19, wherein the order processing rule is based on a timestampassociated with the customer order.
 26. The one or more non-transitorycomputer-readable media of claim 19, wherein the set of rules furtherinclude rules based on an order processing mode, and wherein the orderprocessing mode includes: open plus optional invitation; manualapproval; invitation only; or invitation plus manual approval.
 27. Theone or more non-transitory computer-readable media of claim 19, whereinthe system-related information includes: usage statistics for eachservice of the plurality of services, an amount of storage used for eachservice, an amount of data transferred for each service, a number ofusers, an amount of system up time, or an amount of system down time.28. The method of claim 1, further comprising: determining whether theupdated resource capacity meets the threshold resource capacity; andupon determining that the updated resource capacity meets the thresholdresource capacity, further delaying the request from being sent bypreventing the order information from being removed from the queue. 29.The method of claim 1, wherein the order information is stored andremoved from the queue based on a priority of each of the one or moreorders that are delayed.
 30. The method of claim 29, wherein the orderinformation is removed from the queue based on a priority class of acustomer associated with the order information.
 31. The system of claim10, wherein the computing device is further configured to: determinewhether the updated resource capacity meets the threshold resourcecapacity; and upon determining that the updated resource capacity meetsthe threshold resource capacity, further delaying the request from beingsent by preventing the order information from being removed from thequeue.
 32. The system of claim 10, wherein the order information isstored and removed from the queue based on a priority of each of the oneor more orders that are delayed.
 33. The system of claim 32, wherein theorder information is removed from the queue based on a priority class ofa customer associated with the order information stored within thequeue.
 34. The one or more non-transitory computer-readable media ofclaim 19, wherein the instructions, when executed, further cause the oneor more computing devices to: determine whether the updated resourcecapacity meets the threshold resource capacity; and upon determiningthat the updated resource capacity meets the threshold resourcecapacity, further delaying the request from being sent by preventing theorder information from being removed from the queue.
 35. The one or morenon-transitory computer-readable media of claim 19, wherein theinstructions further cause the order information to be stored andremoved from the queue based on a priority of each of the one or moreorders that are delayed.